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Computing the relative motion of objects is an important navigation task that we routinely 
perform by relying on inherently unreliable biological cells in the retina. The non-linear and adaptive 
response of memristive devices make them excellent building blocks for realizing complex synaptic- 
like architectures that are common in the human retina. Here, we introduce a novel memristive 
thresholding scheme that facilitates the detection of moving edges. In addition, a double-layered 
3-D memristive network is employed for modeling the motion computations that take place in both 
the Outer Plexiform Layer (OPL) and Inner Plexiform Layer (IPL) that enables the detection of 
on-center and off-center transient responses. Applying the transient detection results, it is shown 
that it is possible to generate an estimation of the speed and direction a moving object. 



I. INTRODUCTION 

Computing motion is essential for performing many 
daily tasks that also find applications in machine vision[Tl 
[2], industrial automation [S'-^ and robotics fT', 'S^. Traf- 
fic engineers and transport authorities require motion 
computation and object tracking in order to implement 
effective intelligent transportation systems [TU]. For 
the defense industry, the ability to track mobile enemy 
targetsjTT] and control unmanned vehicles[T2] is critical. 
The tracking of cell movements is also recently shown to 
play an important role in medical diagnostic procedures 
[13l[T4]. While in the field of communications, combining 
motion estimation and tracking with predictive coding 
can allow for significant reduction in transmission band- 
width [T^ . 

By utilizing parallel networks and hierarchical struc- 
tures, biological systems are able to perform motion 
computation very efRcientlv|16|. In comparison, con- 
ventional motion computation systems typically utilize 
charged-coupled device (CCD) cameras and digital pro- 
cessors that result in time-sequential computations. As 
a result, large processing power is required to ensure 
that these serial computations can be performed in real 
time. In recent years, the need for physically smaller 
and more power efficient devices has driven research into 
neuromorphic pn 1 18] and biomimetic systems[19,, which 
are largely inspired by the parallelization naturally found 
in biological systems. 

One of the first biomimetic motion computation sys- 
tems was implemented by a team lead by Carver Mead in 
Caltech[20l El]. By modeling the optical flow via a net- 
work of analog resistive devices, this approach was shown 
to be able to compute an estimate of the optical flow. 
Over the past two decades, various other biomimetic ap- 
proaches to motion computation have been implemented 
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in CMOS technology [201 HUSH- 

Perhaps, the most prominent implementation is a 
128x128 asynchronous temporal contrast silicon retina 
developed by Delbruck[22]. This system makes use of 
address-event representation (AER)[55]: inspired by the 
way neurons utilise spiking events for communication. 
Essentially, the AER approach allows the transmission 
of only the local changes caused by movement in a scene, 
instead of transmitting at a fixed frame rate. This in- 
creases the computational efficiency of the chip, enabling 
it to perform over an impressive dynamic range of more 
than 120 dB with a low power consumption of 23mW. 

Another notable implementation is Visiol pS] , a 
retinomorphic chip with parallel pathways that mimics 
the OPL and IPL. Subthreshold current-mode circuits 
are used to model the autofeedback characteristics of hor- 
izontal cells (for spatial filtering) and the loop-gain mod- 
ulation of amacrine cells (for adapting temporal filtering 
to motion). Detection of edges moving in one direction 
or the other is made possible by aVLSI implementation 
of ganglion cells that respond to motion in a quadrature 
sequence. 

The recent discovery of practical memristive devices 
(3^ has opened the option of improving on these CMOS 
motion sensors. It has been shown in ^37) that memristor- 
MOS technology (MMOST) is capable of outperform- 
ing CMOS implementations in terms of power and size. 
Furthermore, the non-linear and adaptive response of 
memristors allow them to serve as excellent building 
blocks for the practical realisation of complex synaptic 
connections |38l [55] . Our group has previously demon- 
strated that memristive grids have a great potential for 
modeling certain aspects of early vision processing, which 
takes place in the OPL|40|. 

Here, we describe a novel approach, distinct from the 
current implementations (20i i22n34) , that allows for mo- 
tion computation based on memristive networks, by ex- 
tending our results in [40' . A novel memristive threshold- 
ing scheme is utilized for enabling the tracking of moving 
objects within a scene. Also, a biomimetic model of both 
the OPL and IPL are described that facilitates the de- 
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tection of ON and OFF center transient responses. In 
turn, we investigate the directional and speed computa- 
tion capacity of our approach. Finally, we demonstrate 
that this platform can operate in a fault tolerant manner 
and can adapt to distinct lighting conditions, similarly 
to biological counterparts. 



II. BACKGROUND 
A. Vertebrate's Retina: Architecture and Cells 

In order to perform any form of biomimicry, it is nec- 
essary to comprehend the architectural and functioning 
features of the leveraging biological system. This section 
serves as a brief unqualified introduction of the retina and 
is not meant to be a thorough review. More in depth in- 
formation can be found in [4TI - [43| . A simplified drawing 
showing the different layers of the human retina can be 
found in Fig 3 of gH and Fig lof 

There are 5 main types of retina cells (photoreceptor 
cells, bipolar cells, horizontal cells, amacrine cells and 
ganglion cells) laid along two major layers: the Outer 
Plexiform layer (OPL) and the Inner Plexiform layer 
(IPL). The photoreceptors (rods and cones) receive vi- 
sual stimuli from the outside world. The outputs of the 
photoreceptor cells form synaptic connections with bipo- 
lar cells and horizontal cells. The horizontal cells are 
interneurons that form lateral local connections. The 
bipolar cells connect the OPL, comprising of horizon- 
tal and photoreceptor cells, to the IPL containing the 
amacrine cells and ganglion cells. Similar to horizontal 
cells, amacrine cells mediate synaptic lateral interactions 
between bipolar cells and ganglion cells in the IPL. 

The photoreceptor cells essentially act as transduc- 
ing elements, transforming the incident light stimuli into 
electrical signals. The magnitude of voltage change in the 
cells membrane is proportional to the logarithm of the 
intensity of light [22^. Horizontal cells form synaptic con- 
nections between photoreceptors and bipolar cells. Via 
the horizontal cells, lateral connections with neighbour- 
ing groups of photoreceptors and bipolar cells are made. 
These can be conceived as being a resistive layer span- 
ning the OPL, with the architecture allowing dynamic 
range adjustments. 

The output of a bipolar cell depends on the differ- 
ence between the voltage at the photoreceptor it is con- 
nected to and the voltage at neighbouring photorecep- 
tors, through the indirect connection via the horizontal 
cells. There are 2 kinds of bipolar cells. The ON-cells 
that respond to the onset of light and the OFF-cells that 
respond to the cessation of light. The photoreceptor, 
horizontal and bipolar cells do not generate action po- 
tential, instead, computation is performed via the flow of 
depolarization potentials. On the contrary, amacrine and 
ganglion cells in the IPL do generate spikes. Amacrine 
cells give transient light responses to either ON stimulus, 
OFF stimulus or both. Amacrine cells can be classified 



into sustained and transient cells. Sustained amacrine 
cells obtain inputs from the bipolar cells that are not 
inhibited by other amacrine cells. The response of sus- 
tained amacrine cells then inhibits bipolar terminals in 
the narrow field region, causing those terminals to re- 
spond transiently. Essentially, this network of amacrine 
cells forms a high pass filter of the bipolar signal that 
facilitates the detection of moving edges ^45J . 

Ganglion cells carry the output of the retina to the op- 
tical nerve, which leads to the visual cortex in the brain. 
Of the different types of ganglion cells, the directionally 
selective (DS) ganglion cells play a huge role in motion 
detection. There are three identified types of DS gan- 
ghon cells, namely ON/OFF DS ganglion cells, ON DS 
ganglion cells and OFF DS ganglion cells. ON/OFF DS 
ganglion cells perform the function of local motion detec- 
tors. 



B. Mathematical Model of Memristors 

The derivation of the mathematical memristor model 
employed here is largely guided by [35]. A memristive 
system was defined by Chua and Kang[37], following the 
original 1971 definition of the ideal memristor |48j. as: 

V = R{x)i (1) 

^-fM (2) 

This implies that the resistance R{x) depends on an in- 
ternal state X of the device while the time derivative of 
the internal state a; is a function of x and i. The first 
physical model was conceived of a solid-state implemen- 
tation by Strukov et aZ. [49J, where x is proportional to 
charge q flowing through it. In the device, the magni- 
tude of R can be modified reversibly between a highly 
conductive state Ron and a highly resistive state Roff, 
by modulating x: 

R{x)^x{t)Ron + {l~x{t))Roff (3) 

x{t) is restricted to the interval [0,1] and the time deriva- 
tive of x{t) is proportional to the current, as shown: 

Substituting (3) and (4) into (1) we obtain: 

vit)^(3{xit)+r[l-xm^ (5) 

where r — Roff/Ron is the resistance ratio and /3 has a 
dimension of magnetic flux. Moreover, since — J vdt, 
and by using x^ — ^-^x^, it is possible to integrate 
both sides of (5) to obtain: 

r — 1 

(f = /3[ — x"^ + rx + c] (6) 
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where c is a constant of integration determined by the 
initial conditions of x. This equation shows that flux is 
a quadratic function of the charge as x is proportional to 
q, hence, the non-linear relationship |36l [50] . 

Solid-state Ti02 memristors implementations typically 
comprise two metal contacts that encompass a Ti02 ac- 
tive core of thickness D. An external bias across the 
device causes charge to flow through the device. This 
causes a drifting of dopants resulting in the movement 
of the boundary between the two regions. By defining 
w{t) as the coordinates of this boundary and the 
average ion mobility, equation (4) becomes: 



1 dw{t) Ron 



and by integrating both sides, we obtain: 
dw{t) Ron 



D 



fJ-vjy^qit) 



(7) 



(8) 



It is observed that x{t) — w{t)/D corresponds to the 
normalized width of the Ron region while /? = D"^ jp.^. 
Setting c to zero in equation (6) and substituting equa- 
tion (8): 



'^5 = 



l)g' + i?o//(? 



(9) 



Assuming that Rof f»Ron, the memristance of the de- 
vice is obtained as: 



M(g) 



dip 



RoffH-^,) (10) 



Furthermore, as mentioned in |49], there are significant 
non-linearities in ionic transport, especially in the thin 
film edges at the boundary, which can be modeled by a 
window function, /(x), on the right hand side of equation 
(7), giving: 



C. Optical Flow 



(11) 



The estimation of the direction and speed of a pixel 
in a sequence of image can be obtained from the opti- 
cal fiow; the approximation of local motion in an image 
by computing local spatial and temporal derivatives in a 
given sequence of frames. For a 2D picture, the optical 
fiow indicates the speed and direction in which each pixel 
in an image moves between frames. The computation of 
optical fiow is based on spatio-temporal intensity varia- 
tions in brightness patterns. In this work we assume that 
all intensity variations are due to the motion of an object. 
In general, the optical flow and the velocity of an object 
are different. As explained in [51], a perfectly featureless 
rotating sphere has a non-zero velocity component, but it 
will not give rise to any optical fiow. Whereas, a shadow 



moving across the same sphere, which is now stationary, 
will produce an optical flow that is non-zero, although its 
velocity is zero. Nonetheless, apart from these situations, 
the computed optical flow is a good indication of the ve- 
locity of the moving object, provided a strong enough 
gradient between the moving object and its background 
exists. This is generally true for most natural scenes and 
it is assumed to be true for all input stimuli employed in 
this work. 



1. 2D Motion Constrain Equation 

Assuming I(x, y, t) is the intensity of a brightness pat- 
tern at specific pixel located at coordinates (a;,?/) in the 
image at time t, and if this brightness pattern moves by 
{5x, 6y) within a timeframe St, the intensity at the cor- 
responding pixel will be I{x + 6x, y + 5y, t + 5t). Since 
this is essentially the same brightness pattern, it can be 
modeled as: 



I{x, y, t) — I{x + 6x, y + 6y, t + St) 



(12) 



And by employing a Taylor expansion about the point 
(x, y, t), this reads: 



I{x + 5x, y + 5y, t + St) = I{x, y, t) + 



a/ 

dx 



5x 



m 

dy 



Sy - 



9/ 

dt 



St + H.O.T 



(13) 



By comparing equation (12) and (13), it can be derived 
that 



and eventually. 



^Sf — n and ^ ^ 

QfOt -U ana 



dlSy 
dy St 



cUSt 
dt St 



= 



m 

dx 



dl dl 
dy dt 



This can be written more compactly as 

ihjy) ■ («,?i) = -/t = V/-lt 



(14) 



(15) 



V/ refers to the spatial intensity gradient and refers 
to the optical flow at pixel (x, y) at time t. 

Equation (15) has 2 unknowns, which as explained in 
[511 152] stem from the aperture problem. Insufficient in- 
formation is available to measure the full image veloc- 
ity, and only the component perpendicular to the edge 
or along the spatial gradient can be measured. In other 
words, a point in an image sequence only provides one in- 
dependent image measurement whereas the velocity field 
has two components, forming an ill-posed problem. 



2. Smoothness assumption 

In order to solve this ill-poscd problem, a second con- 
strain is therefore required. Horn and Schunck [51 com- 
bined the 2D motion constrain together with a global 
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smoothness term, such the final velocity field it is one 
which minimizes: 

E{U,V) = J J {h+Iy+Itf + 

The first term implies that the solution should be as 
close as possible to the measured data while the second 
term imposes a smoothness constrain on the solution. 
The smoothness constrain derives from the fact that, 
apart from at specific discontinuities, adjacent points on 
an object have similar velocity and the brightness pattern 
varies smoothly almost everywhere. The magnitude of A 
determines the importance of minimizing the smoothness 
term. With high Signal to Noise Ratio (SNR), the im- 
portance of the first term increases and A will be small. 
In contrast, if the data is unreliable, A will be larger, 
emphasizing the importance of the smoothness term. 

Equation (16) shows that E{u, v) is quadratic in the 
unknowns u and v. From standard variation calculus 
it can be shown that the corresponding Euler-Lagrange 
equations are linear in u and v: 

llu + I^IyV - AV^M + Ixh = 

Ijyu + Ilv - XV\ + lylt = (17) 

This results into two linear equations describing every 
point, which essentially capture the necessary compo- 
nents for calculating optical flow. 



III. DETECTING MOVING EDGES WITH 
MEMRISTIVE GRIDS 

We have previously presented (in Figure 4 of ]1^) a 
retinomorphic OPL, which modeled the synaptic inter- 
connections between horizontal, bipolar and photorecep- 
tor cells. Any light stimulus is translated into an appro- 
priate voltage source to bias the underlying hexagonal 
memristive network via a series resistor. It was shown 
that the network was able to perform a local Gaussian fil- 
tering function similar to the biological counterpart [21], 
as well as edge detection via a memristor-based thresh- 
olding scheme. Nodes corresponding to edge pixels have 
large potential difference between neighbouring nodes 
due to the overlying intensity contrast at these pixels. 
From equation (10), the change in the memristance of 
the memristive fuse is proportional to the current flowing 
from that edge node to a neighbouring node. Therefore, 
by monitoring the state variance of memristive fuses as- 
sociated with a node, edge detection can be performed. 
Here, we extend the results of [4^ by performing detec- 
tion of edges of moving objects from a video, investigating 
how a memristive network can be used to detect moving 
edges from a sequence of frames. 



A. Simulation Methods 

The employed video input stimulus is a 80x70 pixel, 
0.6s segment of the video traffic. mju, which can be found 
in the Matlab Image Processing Toolbox[53j. At a frame 
rate of 15fps, this corresponds to 9 frames, shown as a 
montage in Figure la. Changes only occur at interval 
of the frame period(l/15s), and within this period the 
voltage stimulus at each pixel is a constant. This is mod- 
eled with a piecewise-linear voltage source at each node 
corresponding to a pixel. 

Figure lb describes how a single pixel in the video is 
converted into a voltage stimulus. In each gray scale 
frame, the intensity of each pixel is uniformly quantized 
into 256 levels, where represents the absolute black and 
255 represents white. Each pixel is therefore represented 
by an integer from 0-255. Since the maximum voltage in- 
put into each of the voltage input node in the memristive 
grid is set to 40mV, if the quantised intensity of a pixel 
is represented by X, the stimulus to its respective voltage 
input node is X*40/255 mV. This operation is performed 
for each of the pixel in each of the consecutive frames. 
The input voltage stimulus into a specific node corre- 
sponding to a pixel is then set to the piece-wise-linear 
combination of this quantised voltage level. 

MATLAB was employed to implement the netlist for 
the hexagonal memristive network and to convert the in- 
put frames into equivalent biasing voltage stimulus. The 
generated netlist is then imported into HSPICE to per- 
form the necessary circuit simulations on the constructed 
system. Memristive devices were simulated with Bi- 
olek's model [54], with the following parameter settings: 
Ron = lOOn Roff = 16kn R,„,t = 200rj. In addition, the 
Prodromakis's windowing function [SS] was employed to 
take into account the non-linear dopant kinetics of the 
memristive elements. The results from the HSPICE sim- 
ulations were then imported back to MATLAB for fur- 
ther analysis. 



B. Detection of moving edges via memristive state 
thresholding 

The circuit was simulated for the input stimulus shown 
in Figure la. Edge detection was performed for each time 
step of the simulation, using the algorithm described in 
PO] . The results are shown in Figure 2b where the edges 
are marked by a white pixel. We notice that the mem- 
ristance based thresholding scheme described in [40 was 
able to pick up the edges in the first 2 frames. However, 
for the rest of the frames, it seems like that edges were 
being picked up randomly. This is caused by the fact that 
the memristors used for the simulations are non-volatile 
and they do not reset or change before the next input 
frame is being simulated. Hence, even when the next 
frame is being simulated, the past memristance changes 
are still present, causing the results to be corrupted. In 
the following section, we address this issue by incorpo- 
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(a) 



(b) 



FIG. 1. (a) The sequence of the input frames is shown. A specific pixel in the video, marked by a Red X is indicated in each 
of the frames. The time between consecutive frames, Tf, is the inverse of the framerate. (b) The pixel intensity of each frame 
is shown and this corresponds to an input voltage stimulus, which is applied to the respective node in the memristive grid. 




FIG. 2. (a) Evolution of the smoothed input stimulus, (b) 
Simulation of a sequence of images with the edge detection 
algorithm from |40] . Notice that edges were detected success- 
fully only for the first 2 frames. 



rating a novel memristive thresholding scheme that is 
applicable for detecting moving edges. A video showing 
the time evolution of the simulation is provided in SI (see 
Supplementary Material ^56 J. 



Edge detection based on monitoring the 
memristance modulation rate 



rate of change of memristance for the associated memris- 
tive fuses. Hence, by monitoring this rate of change, we 
are able to determine if a node corresponds to an edge 
pixel. 

In a sequence of moving images, once the edge of the 
object moves away from a node, that particular node will 
cease to have a strong intensity gradient with its neigh- 
bouring nodes. Consequently, the temporal derivative of 
the memristance of the associated memristive nodes will 
no longer be high and hence the node will not be denoted 
as an edge. Figure 3 presents results that demonstrate 
our approach. A node is denoted as an edge pixel if the 
rate of memristance change of two or more memristive 
fuse associated with that node exceeds a predetermined 
threshold. Each edge pixel is denoted by a black pixel, 
superimposed on a white background. It is observed that 
the moving edges can now be successfully detected for the 
entire sequence of images. The full transient response of 
this experiment can be found in S2(see Supplementary 
Material[56j). 



To overcome the problem presented presented previ- 
ously, we need to be able to separate the change in mem- 
ristance corresponding to moving edges in the next frame 
from the changes in memristance corresponding to edges 
in the current frame. The proposed solution exploits the 
tracking of the temporal derivative of the memristance 
of the six memristive fuses connected to each node. 

Following Equation (10), the change in memristance 
is proportional to the current flowing through the de- 
vice, which is dependent on the potential difference across 
the device. Since each memristive fuse links adjacent 
nodes, the potential difference across it will be propor- 
tional to the intensity difference between the two nodes. 
If a node is an edge, there will be a strong intensity gra- 
dient with its neighbouring nodes. This results in a large 



(a) 
(b) 

FIG. 3. (a) Evolution of the smoothed input stimulus, (b) 
Moving edge detection by tracking the temporal derivative of 
the memristance of the fuses associated to each node. 



D. Adaptation to varying lighting conditions 

Similar to [ID], we evaluated the proposed algorithm 
against distinct luminance levels. Figure 4 demonstrates 
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that even when the brightness level was halved, the mov- 
ing edges were still detected successfully. The moving 
edge detection results under the two different lighting 
conditions were compared pixel by pixel and the average 
mismatch between these two cases was found to be 4.22%. 
This is illustrated in Figure 4(c), where each white pixel 
corresponds to a mismatch. The full transient response 
of this experiment can be found in S3 (see Supplementary 
Material[55]l. 







(a) 

(b) 











(c) 



FIG. 4. (a) Evolution of the smoothed input stimulus, (b) 
Moving edge detection with input stimulus of half the original 
brightness, (c) Illustrates the accentuated mismatch between 
moving edge detection under normal lighting condition and 
under half the original brightness. 



E. Memristance Variability Tolerance 

The memristive network was tested to determine how 
tolerant it was to potential defects in the network. To 
simulate this, defect memristors were artificially intro- 
duced. Instead of simulating a circuit with all uniformly 
initialised memristors (i.e. i?o„ — lOOil Roff — IGkil 
Rinit = 200ri), we now initialise a percentage of the 
memristors randomly. These defected memristors will 
be initialised randomly with Ron — 50Q — lOOil in in- 
crements of lOil, Roff — lOfcil — 20kn in increments of 
O.bkil, Rinit as a multiple (2-20) of the new Ron- The 
simulation results for 50% defects and 80% defects are 
respectively shown in Figure 5b and 5c. 

The memristive network appears to be rather robust 
to faults, even when 80% of the devices were initialised 
randomly. The full transient response of this experiment 
can be found in S4(see Supplementary Material |56|). The 
results presented in Figure 5 demonstrates that the pro- 
posed system was still able to perform the detection of 
moving edges reliably. This is largely attributed to the 
fact that there are inherent redundancies in the network, 
as each node is interconnected to 6 neighbouring nodes. 
This is particularly important as the fabrication yield of 
memristive devices is relatively poor given that the mem- 
ristors technological readiness is rather low. The scalabil- 
ity of memristive technology indeed opens the possibility 
of achieving reliability through redundancy. 




(c) 

FIG. 5. (a) Evolution of the smoothed input stimulus, (b) 
Moving edge detection with artificially simulated faults of 
50%. (c) Moving edge detection with artificially simulated 
faults of 80%. 



IV. DOUBLE LAYERED MEMRISTIVE 
NETWORK 

In the previous section, it was demonstrated that by 
employing the proposed thresholding scheme, memristive 
networks were able to perform moving edge detection, 
even with artificially simulated faults and at varying lu- 
minance levels. At the same time, this implementation is 
able to achieve local Gaussian filtering with a lower com- 
plexity as compared to other systems [401 and has been 
shown to perform better than resistive grids for edge pre- 
serving smoothing '57' . However, with this implementa- 
tion, full optical flow computation is still not quite pos- 
sible, as we have not yet obtained enough information 
from the sequence of moving images. The derivation of 
optical flow (in section II. C) shows that Equation (17) 
has to be solved in order to obtain the required optical 
flow. This requires both the spatial derivatives and the 
temporal derivatives at each pixel but our current im- 
plementation only provides us the spatial derivative (i.e. 
through the rate of change of memristance of memristive 
fuses). In this section, we present a 3D memristive grid 
that facillitates the computation of the temporal deriva- 
tive. 



A. Biomimetic OPL and IPL 

From the results in the previous section, we note 
that a second memristive layer is required to model the 
amacrine cells within the IPL of the retina that are re- 
sponsible for generating the transient responses. Our 
model does not explicitly emulate the functioning of the 
neurons in the IPL, but rather the synaptic interconnec- 
tions between them. In order to generate the transient 
responses, a delay of the input stimulus is required, and 
this is introduced by the delayed version of the input in 
the second layer. 

The 3D network, part of which is illustrated in Fig- 
ure 6, consists of 2 hexagonal memristive layers intercon- 
nected with memristive fuses at each node. The current 
signal is applied to layer 1, which is responsible for com- 
puting the edges from the sequence of frames. Whereas, 
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FIG. 6. Illustration of a section of the proposed 3D hexago- 
nal memristive network model. Each node corresponding to a 
specific pixel is connected in a hexagonal manner to 6 neigh- 
bouring nodes via memristive fuses. The network consists of 
two interconnected layers. The current signal is applied to 
Layer 1 while a delayed signal of 1 frame is applied to layer 2. 
Each pixel in the input frame is converted into a piecewise- 
linear voltage value, which is then applied through a resistor 
to the memristive grid. For clarity, only two of these biasing 
connections are shown. 

in layer 2, a delayed version of each frame is used, en- 
abling the computation of the temporal derivative at each 
node via the memristance changes in the interconnecting 
memristive fuses. 

Each connecting memristive fuse is made up of two 
individual memristors. We label the two memristor in 
each memristor fuse as Connecting Memristor 1 (cMl) 
and Connecting Memristor 2 (cM2), where cMl is the 
memristor closer to layer 1 and cM2 is the one closer to 
layer 2. 

When an object in the input stimulus is completely sta- 
tionary, it implies that the pixels corresponding to that 
object remains the same for the current frame n and the 
delayed frame (n-1). Consequently, the spatial voltage 
input (representing this object) to each layer will be iden- 
tical and both ends of all connecting memristors are at 
equipotential. No current would thus flow and there will 
be no observable change in memristance. 

Conversely, when an object in the input stimulus 
moves, the pixels corresponding to that object will move. 
This means that the current frame n is slightly different 
from the delayed frame (n-1), reflecting this movement. 
Consider the case where a black box moves to the right 
by 2 pixels on a white background, as shown in Figure 7a. 
It can be observed that the rightmost and leftmost black 



edges of the box will be moving by 2 pixels to the right. 
Comparing with frame (n-1), the black box in frame n 
will be a shifted version by 2 pixels to the right. Hence, 
there will be a potential difference between the nodes cor- 
responding to the new rightmost black edge (in frame n) 
and the old leftmost black edge (in frame (n-1)). At the 
new rightmost black edge, there will be a higher potential 
at layer 2 compared to layer 1. This causes a current flow 
through the connecting memristor fuse in from layer 2 to 
layer 1. Therefore, it is expected that the memristance 
cM2 will be increasing while the memristance of cMl will 
be decreasing. Whereas for the old leftmost black edge, 
there will be a higher potential at layer 1 compared to 
layer 2. A current will flow from layer 1 to layer 2 causing 
the memristance of cMl to increase and cM2 to decrease. 

Therefore, by monitoring the memristance changes in 
each memristor of each connecting memristor fuse, it is 
possible to determine the movement of the edges. For 
a dark object moving on a light background, decreas- 
ing cMl and increasing cM2 corresponds to an appear- 
ing edge while increasing cMl and an decrease of cM2 
corresponds to a disappearing edge. This is similar to 
the on-center response in the retina. On the other hand, 
for a light object moving on a dark background, increas- 
ing cMl and decreasing cM2 corresponds to an appear- 
ing edge while decreasing cMl and increasing cM2 cor- 
responds to a disappearing edge. This is similar to the 
off-center response of the retina. Essentially, the response 
of these connecting memristors captures the response of 
the transient amacrine cells in the retina. 

With a delayed version of the input into the second 
layer of memristive network, it is possible to derive the 
temporal derivative between consecutive frames. The 
temporal derivative between corresponding pixels in con- 
secutive frames is inherently computed as changes in 
memristance of the memristors within the connecting 
memristive fuse. By combining the temporal derivative 
information with the spatial derivative information ob- 
tained from the hexagonal memristive network, it will be 
possible to separate moving objects from stationary ob- 
jects. Essentially the hexagonal grid is performing com- 
putation of the sustained response while the connecting 
memristors are responsible for computing the transient 
response. 



B. Emulating Transient Detection 

1. Simple Inputs 

As an initial proof of how the abovementioned motion 
computation can be done with the double-layered mem- 
ristive network, two simple simulations were performed. 
Each of the simulation input stimulus consists of a Is, 15 
fps, 17x30 pixels video clip. 

The first simulation, illustrated in Figure [7] involves 
a black box moving across a white background. From 
frame 1 to frame 5, the box is moving to the right at a 
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FIG. 7. (a) Evolution of input stimulus: A black box moving to the right at a rate of 2 pixels per frame from framel to frame 
5, and 1 pixels per frame from frame6 to frame 15. (b) Simulation result of moving edge detection, (c) OFF-center transient 
detection results from monitoring the connecting memristive fuses, (d) ON-center transient detection results from monitoring 
the connecting memristive fuses. 



rate of 2 pixels per frame and from frame 6 to frame 15, 
the box contimies moving to the right at a slower rate 
of 1 pixel per frame. Since the moving object is dark 
compared to the surrounding, the on-center transient re- 
sponse (Figure 7(d)) is zero, which is expected. In the 
ofF-center transient response (Figure 7(c)), the light grey 
pixel corresponds to the appearing edge, while the dark 
pixel corresponds to disappearing edges. We see that 
the model is capable of monitoring the movement of the 
edges, while the speed of this movement can be derived 
from the thickness of the transient edges. 

Our second example is similar to the first except for 
the fact that instead of a black box moving across a 
white background, a white box is now moving across 
a black background. In this case, the off-center tran- 
sient response is zero and all the transient response is 
due to the on-center transient response. Here, the dark 
grey pixels correspond to an appearing edge while the 
light grey pixels correspond to a disappearing edge. The 
results are further illustrated in S5(see Supplementary 
Material[56i|). 

2. Complex Inputs 

The simulation was extended to a more complex input, 
which involves the first two frames of the traffic video 
stimulus shown in Figure la. Only twp frames were 
simulated as the simulation of more frames are rather 
computationally intensive. The frame size used in this 
simulation was 80x70pixel. This results in a circuit with 
22400 nodes, 38602 memristive fuses, 11200 resistors and 
11200 varying voltage sources. Even when running on a 
HP Z820 workstation with two 8-core Intel Xeon 2Ghz 
processor and 64 GB of RAM, the simulation environ- 



ment (HSPICE) is incompetent in solving more extended 
cases. 

The input stimulus, shown in Figure 8(a) consists of 2 
frames. Frame 2 is overlaid on Frame 1, and for clarity, 
is shifted slightly to the right and it has purple colour 
saturation. From Frame 1 to Frame 2, it can be observed 
that the 2 white cars have moved slightly forward down 
the road, while the black car has almost moved off the 
image. The simulation was performed for 1 frame period 
of 1/15 s. By monitoring the memristance changes of the 
memristors within the connecting fuse, the results are 
shown on Figure 8(c) and (d) are obtained. From Fig- 
ure 8(b), we do not consider the effects of the on-center 
or off-center response. The lighter grey dots correspond 
to decreasing cMl and increasing cM2 while the darker 
grey dots corresponds to the increasing cMl and decreas- 
ing cM2. This result shows the computation of transient 
response, where only moving objects were picked up. 

In order to accurately compute the direction and speed 
of the object, it is necessary to separate the movement of 
dark edges from the movement of light edges. Essentially, 
the on-center response and the off-center response have 
to be distinguished. This can be done by splitting the 
result in acquired in Figure 8(b) based on the intensity 
of the input stimulus to each pixel. The off-center re- 
sponse is shown in Figure 8(c), where only the transient 
response due to a moving dark object/edge is shown. The 
darker dots correspond to a disappearing dark edge while 
the lighter dots correspond to an appearing dark edge. 
Similarly, the on-center response, showing the transient 
response due to a moving light object/ edge is shown in 
Figure 8(d). The darker dots now correspond to an ap- 
pearing light edge while the lighter dots correspond to a 
disappearing light edge. 



9 




(a) 




(b) 




FIG. 8. Simulation of 2 frames from a video of traffic on 
a fiigfiway. (a) Input stimulus consists of 2 frames. Frame 2 
is overlaid on Frame 1, and for clarity, it is shifted slightly 
to the right and it has purple colour saturation (b) Transient 
detection (c) OFF-Center response (d) ON-center response 



C. Directional and Speed detection 

Given the output from the transient detection, it is 
possible to use a 1-layer hexagonal memristive grid to 
compute the direction and speed of the moving object. 
The results from the transient detection shown in Figure 
8(c) and (d) can be used as the input stimuli. To demon- 
strate this, the memristive grid was biased based on the 
the off-center response from Figure 8(d). The darker pix- 
els will correspond to a positive voltage source into the 
memristive network while the lighter pixels will corre- 
spond to a ground,. Figure 9(a) illustrates the 80x70pixel 
hexagonal memristive grid where the voltage sources and 
ground are marked by "x"and "o" respectively. The rate 
of memristance changes of all interconnecting fuses are 
monitored and they are drawn in Figure 9(a) with a value 
dependent colour variation. Since it is expected that the 
current will take the shortest path from the source to 
the ground, the rate of memristance changes of the fuses 
will provide an indication of the direction and flow of the 
current. At each of the moving edge pixel, an estimate of 
the optical flow can be computed by the vectorial sum of 
the rate of memristance change of the 6 interconnecting 
fuses. Figure 9(b) and 9(c) respectively shows the results 
of this computation for fnset (a) and Inset (b) of Figure 
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FIG. 9. (a) Results illustrating the rate of memristance 
change of fuses within the 80x70pixel network. Each mem- 
ristive fuse is plotted with a color corresponding to the rate 
at which it's memristance changes, (b) Illustrates the optical 
flow estimate from Inset (a), (c) Illustrates the optical flow 
estimate from inset (b). 



9(a). Our method provides local optical flow estimates 
at the brightness edges, and as illustrated in Figure 9(b) 
and 9(c), was able to correctly determine that both cars 
were moving downwards. 



V. CONCLUSION 

In this paper, we presented a novel method for com- 
puting motion by leveraging on the scalability and non- 
linearity of memristors for emulating both the OPL and 
IPL of a vertebrates retina. The parallel processing na- 
ture of the proposed memristive network offers great po- 
tential in minimizing the computation and power require- 
ments of conventional motion detection systems. A novel 
memristive thresholding scheme was introduced to allow 
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the detection of moving edges. In addition, a double- 
layered memristive network was constructed to enable 
to detection of ON-center and OFF-center transient re- 
sponses. Applying the transient detection results into a 
1-D memristive network, it was then shown that it is pos- 
sible to generate an estimate of the speed and direction 
of the moving object. This study serves as a good proof 
of concept that memristive networks have indeed great 



potential in performing motion computation effectively 
and efficiently. 
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